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PREFACE 


The Agriculture and Resources Inventory Surveys Through Aerospace Remote 
Sensing Is a program of research, development, evaluation, and application 
of aerospace remote sensing for agricultural resources. This program Is a 
cooperative effort of the National Aeronautics and Space Administration, 
the U.S. Departments of Agriculture, Commerce, and the Interior, and the 
U.S. Agency for International Development. 

The work which Is the subject of this document was performed by the Earth 
Resources Applications Division, Space and Life Sciences Directorate, 

Lyndon B. Johnson Space Center, National Aeronautics and Space Administration 
and the Lockheed Engineering and Management Services Company, Inc. Tasks 
performed by Lockheed were accomplished under Contract NAS 9-15800. 
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1 . EXECUTIVE SUMMARY 

The 1980 U.S. /Canada Wheat and Barley Exploratory Experiment was designed to 
further develop state-of-the-art area estimation technology and test It In a 
foreign similar environment. Pre-AgRISTARS research had Identified technical 
Issues In the reliability and efficiency of estimating spring small grains In 
the U.S. /Canada Northern Great Plains environment and In the separation of 
spring wheat from spring barley using remote sensing data. Approaches had 
been developed which provided potential Improvement for solving these Issues. 
This experiment was oriented to develop and test these approaches for potential 
further testing and development leading to foreign application. 

t 

Developmental activities were Initiated to produce an advanced technology 
which was not only accurate but efficient and objective. The Improvements 
were directed towards developing an automated area estimation technology with 
minimal analyst Interaction as one component of a foreign commodity production 
forecasting system. 

In response to these objectives, the Foreign Commodity Production Forecasting 
(FCPF) and Supporting Research (SR) projects developed Improved crop Identifi- 
cation procedures, machine processing techniques, and crop calendar models. 

The FPCF Project Integrated this technology Into the area estimation system 
and Implemented the exploratory test and evaluation. The exploratory evalua- 
tion was conducted in order to better understand the performance of this 
newly developed technology before proceeding to a pilot experiment for evalua- 
tion under a larger and more varied set of agricultural and environmental 
conditions. 

The techniques developed and Integrated Into the FCPF developmental area 
estimation component for evaluation during the 1980 U.S. /Canada Wheat and 
Barley Exploratory Experiment were: (1) objective crop Identification proce- 

dures designed to produce consistent and accurate spring small grains Identi- 
fication/labeling results, (2) advanced machine processing techniques developed 
to Improve the estimation of crop area within the sample segments, and (3) 
recently developed crop calendar models designed to provide improved estimates 
of the crop development stages for wheat and barley. 
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The results of the experiment indicated that the new crop identification proce 
dures performed well for spring small grains and that they are conducive to 
automation. The performance of the machine processing techniques shows a 
significant improvement over previously evaluated technology. However, the 
crop calendars will require additional development and refinements prior to 
integration into automated area estimation tec’inology. 

The evaluation has shown the integrated technology is capable of producing 
accurate and consistent spring small grains proportion estimates. However, 
barley proportion estimation technology was not satisfactorily evaluated. 
Landsai. sample segment data was not available for high-density barley of 
primary importance in foreign regions. The low density segments examined 
were Judged not to give indicative or unequivocal results. 

It is concluded that, generally, the spring small grains technology is ready 
for evaluation in a pilot experiment focusing on sensitivity analyses to a 
variety of agricultural and meteorological conditions representative of the 
global environment. It is further concluded that a strong potential exists 
for establishing a highly efficient technology for spring small grains. 
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2. PURPOSE AND SCOPE 


The purpose of this report is to summarize the results from the 1980 U.S./ 
Canada Wheat and Barley Exploratory Experiment. The developmental activities 
and experiments reported in this document cover activities of the AgRISTARS 
Foreign Coninodity Production Forecasting Project. These activities include 
component-level exploratory development, integration and testing of crop 
identification procedures, alternative computer classification techniques 
and candidate crop development stage models. Remote sensing research related 
to wheat and barley has also been conducted by the Environmental Research 
Institute of Michigan (ERIM) for the AgRISTARS Supporting Research Project 
and is reported elsewhere (Ref. 1). 
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3. INTRODUCTION 


3«1 AqRI STARS PROGRAM 

The Agriculture and Resources Inventory Surveys Through Aerospace Remote 
Sensing (AgRISTARS) program is a 6-year program of research, development, 
and evaluation of the application of aerospace remote sensing to monitoring 
agricultural resources. The progrsn began in fiscal year (FY) 1980. The 
AgRISTARS program is a cooperative effort of the National Aeronautics and 
Space Administration (NASA), the U.S. Departments of Agriculture, Commerce, 
and the Interior (USDA, USDC, and USDI), and the U.S. Agency for International 
Development (AID). The goal of this program is to determine the usefulness, 
cost, and extent to which aerospace remote sensing data can be used by the 
USDA to improve the objectivity, reliability, and timeliness of information 
required to carry out USDA missions (Ref. 2). 

3-2 FCPF PROJECT 

The objective of the Foreign Commodity Production Forecasting (FCPF) project 
is to develop and test procedures for using aerospace remote sensing technology 
to provide more objective, timely and reliable crop production forecasting in 
foreign areas. To develop technology for use in foreign areas, the FCPF project 
builds upon existing remote sensing technology, and extends this technology to 
additional crops and regions. 

During the first year of the FCPF project, two exploratory experiments were 
performed using U.S. data to develop and evaluate techniques. These experi- 
ments were the U.S. /Canada Wheat and Barley Exploratory Experiment (1) and the 
U.S. Corn/Soybean Exploratory Experiment. This report presents the results 
from the U.S. /Canada Wheat and Barley Exploratory Experiment (Ref. 3). 

3.3 U.S. /CANADA WHEAT AND BARLEY EXPLORATORY EXPERIMENT 

The overall objective of the 1980 U.S. /Canada Wheat and Barley Exploratory 
Experiment was to develop, test and evaluate state-of-the-art technology for 
spring small grains, wheat and barley to establish a basis for further develop- 
ment of estimation technology to be applied in foreign regions, specifically 
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the U,S.S,R., and indirectly, to Australia and Argentina. The technical 
emphasis for this exploratory experiment was to; 

• Develop accurate and objective crop identification/labeling 
techniques (Ref. 4). 

• Develop a machine processing technology with improved performance 
characteristics (Ref. 5). 

e Develop alternative crop calendar/crop development stage models for 
making improved estimates of wheat and barley development (Ref. 6). 
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4. EXPERIMENT DESCRIPTION 


Three tests were performed as part of this exploratory experiment. The first 
test was the labeling p^sjcedures test. The second was the evaluation of 
machine processing/classification technology. The third test was the crop 
calendar/crop development stage models test. The functional flow of a concep- 
tual system which has these components incorporated into it is shown in 
Figure 1 . 

4.1 LABELING PROCEDURES TEST 

The labeling procedures test was designed to test and evaluate a newly devel- 
oped objective labeling procedure. The test was conducted in two phases, 

• Phase 1 - A shakedown test using six 1978 segments. 

§ Phase 2 - An expanded test using 35 segments from a different 
crop year (1979). 

The locations of the segments used in the test are shown in Figures 2 and 3. 

The objectives of this test were: 

t To determine the accuracy and objectivity of the newly developed 
spring small grains labeling procedure. 

• To determine the accuracy of the barley estimation technology. 

In both phases of the test, an objective labeling procedure (Ref. 7) was used 
to label Landsat pixels in each segment. Input data to the procedure consisted 
of Landsat multispectral scanner data, crop calendar information, and ancillary 
agronomic/meteorological data. 

The procedure is designed to provide increasingly detailed labeling information 
at each step using a tree-structured decision logic (Fig. 4). The first step 
consists of a labeling logic which is used to separate the pixels into cropland 
and noncropland. The pixels labeled cropland in the first step are separated 
into spring small grains and other crops in the second step. In the third 
step, Landsat spectral aids are used for separating the spring small grains 
into barley and other spring small grains. 
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Figure 1. - A Functional Flow of the FCPF Area Estimation Approach 













FIGURE 2. - 1978 SEGMENT LOCATIONS FOR THE LABELING PROCEDURES SHAKEDOWN TEST 
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Figure 4. Diagram showing the major steps in the objective ! 

labeling logic which leads to identification of | 

barley and other spring small grains. i 
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The segirents in the Labeling Procedure Test were processed, independently, by 
two analysts in order to evaluate the repeatability and objectivity of the 
procedure. The evaluations were performed by comparing the labeling results 
from the Integrated Analysis Procedure (Ref. 8) to the segment ground truth 
inventories. An error characterization study was performed to determine if 
any changes to the labeling procedure were required to improve the objectivity 
or accuracy. 

4.2 MACHINE PROCESSING/CLASSIFICATION TEST 

The machine processing/classification test consisted of processing and 
classifying the same 1979 U.S. Spring Wheat Region segments used in the 
Labeling Procedures Test. The objective of the test was to evaluate the 
accuracy and efficiency of alternative classification techniques. 

A need for more efficient use of labeled samples in segment proportion estima- 
tion had previously been established by studies which showed that simple 
random sampling could produce results equivalent to maximum likelihood classifi 
cation. The Supporting Research (SR) Project developed a Bayes approach to 
proportion estimation using a stratified sample in response to this deficiency 
(Ref. 9). This technique was integrated with the labeling procedure to form a 
proportion estimation component. It was included in the exploratory experiment 
for evaluation. 

The following alternative techniques for allocating samples and estimating 
crop proportion within each segment were evaluated (Ref. 10). 

• Random Sample/Relative count - this technique allocates samples 
randomly and estimates crop proportions by determining the number 
of samples in a crop category and dividing by the total number of 
samples. 

• Proportional Allocation/Relative Count - samples are allocated to 
clusters proportional to the cluster sizes and the estimate is 
generated by determining the number of samples in a crop category 
per cluster and weighing the estimate by cluster size. 
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• Proportional Allocation/Bayes Estimator - the samples are allocated 
to clusters proportional to cluster size and proportion estimation 
is calculated using the Bayes Estimator. 

t Bayes Sequential Allocation/Bayes Estimator - samples are allocated 
to clusters sequentially in an attempt to minimize the mean square 
error (MSE) and a proportion estimate is calculated using the 
Bayes Estimator. 

In the last three evaluations, the samples were stratified using the CLASSY 
clustering algorithm (Ref. 11, 12, 13). 

4.3 CROP DEVELOPMENT STAGE MODEL TEST 

The crop development stage model test consisted of estimating the planting date 
and phenological development stages of wheat and barley in 49 segments within 
the U.S. Spring Wheat Region. Figure 5 shows the location of the segments used 
in the test. 

The objectives of this test were: 

t To evaluate alternative models. 

• To determine which combination of planting date and phenological 
development stage models most accurately estimate the development 
of wheat and barley. 

• To determine if the various models are sufficiently accurate to be 
incorporated into objective labeling procedures. 

The models evaluated in this test are: 

• Planting-date models tested 

- Normal Planting-date Model (Ref. 14) 

- Feyerherm Planting-date Model (Ref. 15) 

t Wheat Phenological Development Stage Models tested 

- Original Robertson Wheat Model (Ref. 17) 

- improved Robertson Wheat Model, Version 1 (Ref. 15) 

- Improved Robertson Wheat Model, Version 2 (Ref. 15) 
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• Barley Phenol ogical Bevel opnsent Stage Model tested 
- Williams Barley Model (Ref. 16) 

The Feyerherm and the Normal planting-date models were evaluated on their 
ability to accurately predict the median planting dates in the segments. The 
basis for comparison was the ground truth median planting dates. The ground 
truth median planting dates for spring wheat and barley were obtained by 
calculating the date at which 50 percent of the spring wheat and barley fields 
in each of the segments were observed to be planted. Discrepancies between 
ground truth and the models were measured in number of days. 

The performances of the three Robertson growth stage models were evaluated 
using the ground truth median growth stages as the basis for comparison. 
Observed median planting dates were used to initiate the models. The ground 
truth median growth stages for spring wheat and barley were obtained by 
calculating the observed median stage for spring wheat and barley fields 
within each of the segments for each of the dates on which the stages were 
observed. The comparison of the models' prediction versus the observed crop 
stage yielded errors in terms of crop stages associated with each of the 
model s . 

The barley growth stage model was evaluated using the observed median planting 
dates for barley to initiate the models and subsequently comparing the model 
prediction of stage with the ground truth median growth stages for barley. 
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5. EXPERIMENT RESULTS 


5.1 LABELING PROCEDURES TEST RESULTS 

The shakedown test of the objective labeling procedure using 1978 Landsat 
data Indicated: 

• Excellent Spring Small Grains labeling accuracy results. The 
overall accuracy was 76%. 

• Labeling results were comparable to an analyst Intensive procedure 
performed on 1978 data (76% versus 75%). 

• Consistency between the analysts was very good. Overall, the 
agreement on labels was 85%. 

The expanded labeling test using the 1979 data provided the following results: 

• Labeling accuracy results for spring small grains were similar to 
the Integrated Analysis Procedure, although slightly lower, 66% 
for the objective labeling procedure versus 76% for the Integrated 
Analysis Procedure. 

• The 1979 error characterization study Identified the areas requiring 
improvements to the objective labeling procedure. 

- The procedure processed only 25% of the available segments. 

- Confusion of pasture with small grains was a problem. 

- Crop calendar improvements were required in order to better 
select acquisitions for processing. 

The wheat/barley separation procedure results are: 

t Segments with 10% and above barley were not available for testing. 

• Segments were not available which have both winter wheat and 
spring barley as In the foreign similar environment. 

e The labeling accuracy was approximately 50% in low density barley 
segments; those containing 5% or less. 
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5.2 MACHINE PROCESSING/CLASSIFICATION TEST RESULTS 

Results of the machine processing/classification test based upon ground truth 

input labels are: 

t A significant increase in the precision of segment proportion 
estimates was obtained by CLASSY stratification (Table 1). 

- This was the first time a machine processing technique had 
performed better than the technique of using simple random 
sampling and making the proportion estimate by relative count. 

- It requires 3 times as many labeled pixels for a randomly 
sampled segment in order to achieve same proportion estimation 
precision as when CLASSY stratification is used. 

• Segment proportion estimation bias and mean square error (MSE) 
are significantly reduced by machine processing/CLASSY stratifica- 
tion when compared with the results from random sampling (Table 1). 

• There is not a significant difference in the performance of thethree 
machine allocation and estimation techniques; (1) proportion allo- 
cation/Relative Count, (2) proportional allocotion/Bayes Estimator, 
ond (3) Bayes Sequential allocation/Bayes Estimator. 

5.3 CROP DEVELOPMENT STAGE MODEL TEST RESULTS 

The results from the test of planting-date models are; 

• The Feyerherm Model is significantly better than the Normal Model 
for predicting both spring wheat and barley planting dates. 

(Fig. 6a and 6b, respectively) 

Results from the test of the Wheat Phenological Development Stage Models are: 

• There are no significant differences between the 3 models (Original 
Robertson and the two improved versions) in estimating the develop- 
ment stages from tillering to ripening. 

t The improved Robertson Models, versions 1 and 2, appear to estimate 
the late heading and ripening stages of wheat more accurately than 
the original Robertson Model. 
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Figure 6a 


PLANTING DATE MODEL RESULTS 
FOR SPRING WHEAT 


* OVERALL tTATItnCt INDICATE THE FEYERHERM MODEL II CLOtER 
TO GROUND TRUTH THAN NORMAL MODEL IN PREDICTING SPRING 
WHEAT PLANTING DATES. 


DISTRIBUTION OF ERRORS (IN DAYS) FOR THE FEYERHERM VS. THE NORMAL PLANTING 
DATE MODEU APPLIED TO SPRING WHEAT 
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Figure 6b 


PLANTING DATE MODEL RESULTS 

FOR BARLEY 


* OVERALL ETATItTICt INDICATE THAT FEYERHBRM !• 
CLOEER TO THE GROUND TRUTH THAN THE NORMAL 
IN FREDICTINO BARLEY PLANTING DATE!. 


OIITRIBUTION OF ERROR! (IN DAYS) FOR THE FEYERHERM VI. THE NORMAL PLANTING 

DATE MODELS APPLIED TO BARLEY 
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Results from the test of the B«rley Phenological Development Stage Models are: 

• The Robertson Spring Wheat Models performed better than the 
Williams Barley Model . 

None of the models predicted the wheat/barley separation period very accurately. 
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6. CONCLUSIONS AND RECOMMENDATIONS 

The results from the Spring Small Grains labeling procedures evaluation 

Indicate; 

f Labeling results are comparable to the Integrated Analysis 
Procedure. 

• The procedure Is conducive to automation. 

• Error sources are easily Identified and quantified due to the 
tree-structured C:^s1gn of the procedure. 

• Improvements to the labeling procedure are required to eliminate 
the confusion of pasture and spring small grains. 

t Additional criteria for defining acceptable Landsat acquisitions 
for processing are required. 

Results from the wheat/barley separation evaluation are; 

• The labeling accuracy was approximately 50% In low-density barley 
segments. 

• Because high-density barley segments were not available, the proce 
dure was not adequately evaluated. 

• Crop development stage models were Insufficient for selecting the 
wheat/barley separation acquisition. 

The machine processing/classification procedure results Indicate; 

• CLASSY stratification Improved the precision of the proportion 
estimation procedures. 

• Estimation bias and the mean square error (MSE) were significantly 
reduced over random sampling for the first time ever. 

The crop development stage model test results indicate; 

t The Feyerherm Planting Date Model performs better than the Normal 
Model for both spring wheat and barley. 
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• The performance of all three versions of the Robertson Spring 
Wheat Model is similar. 

• The performance of the Robertson and Feyerherm Models appears to 
be satisfactory for integration into automated labeling proce- 
dures; however, further evaluation is recommended. 

In summary, it is concluded that: 

• The results of the exploratory experiment indicate a strong poten- 
tial for establishing the basis for a highly efficient technology 
for evaluation in a foreign environment. 

• A pilot experiment should be conducted in order to further develop, 
test, and evaluate the technology prior to conducting a foreign 
pilot. The pilot should be conducted on spring small grains in 
the U.S. /Canada Northern Great Plains Region. 

• Technology development should focus on the development and eval- 
uation of techniques for efficient area estimation technology, 
sensitivity analysis of spring small grains area estimation 
procedures, and assessing performance to be expected in foreign 
countries. 
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